Probabilistic and Sequential Computation of Optical Flow Using Temporal Coherence * ( revised version of LIDS - P - 2122 )

In the computation of dense optical flow fields, spatial coherence constraints are commonly used to regularize otherwise ill-posed problem formulations, providing spatial integration of data. In this paper we present a temporal, multi-frame extension of the dense optical flow estimation formulation proposed by Horn and Schunck [1] in which we use a temporal coherence constraint to yield the optimal fusing of data from multiple frames of measurements. Conceptually at least, standard Kalman filtering algorithms are applicable to the resulting multi-frame optical flow estimation problem, providing a solution which is sequential and recursive in time. Experiments are presented which demonstrate that the resulting multi-frame estimates are more robust to noise thani those provided by the original, single-frame formulation. Also, we demonstrate cases where the aperture problem of motion vision cannot be resolved satisfactorily without the temporal integration of data enabled by the proposed formulation. Practically, the large matrix dimensions involved in the problem prohibit exact implementation of the optimal Kalman filter. To overcome this limitation we present a computationally efficient, yet near optimal approximation of the exact filtering algorithm. This approximation has a precise interpretation as the sequential estimation of a reduced order spatial model for the optical flow estimation error process at each time step and arises from an estimation-theoretic treatment of the filtering problem. Experiments also demonstrate the efficacy of this near optimal filter.